Knowledge and Data Fusion in Probabilistic Networks

Probability theory provides the theoretical basis for a logically coherent process of combining prior knowledge with empirical data to draw plausible inferences and to refine theories as observations accrue. Increases in the expressive power of languages for expressing probabilistic theories have been accompanied by refinements and adaptations of Bayesian learning methods to handle the more expressive constructs. These innovations have established Bayesian learning as a unifying theoretical framework for learning in intelligent systems, and have given rise to practical techniques that are receiving wide application. This paper describes theory and methods for exact and approximate learning of probabilistic theories from a combination of background knowledge and observations. The concepts and methods can be adapted to any knowledge representation framework that can express probability distributions over interpretations of a first-order logic. We focus specifically on methods to learn theories that can be expressed in the Multi-Entity Bayesian Network (MEBN) probabilistic logic. MEBN logic is sufficiently general to represent a probability distribution over interpretations of any set of statements that can be expressed in firstorder predicate calculus. Bayesian inference provides both a proof theory for combining prior knowledge with observational evidence to derive plausible conclusions and a learning theory for refining a representation as observational evidence accrues. A formal specification is provided for the MEBN logic. A semantics is based on random variables provides a logically coherent foundation for open world reasoning. The paper describes modifications of standard Bayesian learning methods to handle the repeated structures that occur in MEBN theories. Methods are given for specifying domain knowledge as MEBN fragments with structure and parameter prior distributions.